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EDUCATIONAL GOALS: HOW MUCH AGREEMENT CAN Iiffi GET 

on mcH ONES are iMPOirrANT? 



Ray L. Sweigert, Jr. 
Atlanta Public Schools 

Objectives , Objectives were: (l) to establish goals for education 
in Atlanta, using the Delphi technique; (2) to determine the extent to 
which different groups— coiiimunity leaders, educators, and students, 
broken down by r^ce and sex — converged toward agreement on the relative 
importance of goals in the Delphi studies; and (3) to identify the goals 
that constituted the most critical areas of disagreement among these 
groups. 

The first objective was a major imdertaking of the Atlanta Assess- 
ment Project (AAP). AAP is a three-^year endeavor to develop techniques 
and tools for measuring the progress of Atlanta's 17 and l8-year^ld 
youth, both in school and out, toward the achievement of educational 
goals relevant to living in the Atlanta of I985 and thereafter. Admini- 
stered and operated within the Atlanta Public Schools, the project is 
funded under Title III, ESEA. The second and third objectives above 
were subordinate widertakings of the AAP. It is these latter two objec- 
tives, however, that are the primary focus of this paper. 

Theoretical Framework . It is generally recognized that there are 
two types of forecasting involved in establishing educational goals. One 
type forecasts what conditions probably will be at a given time in the 
future, and the other forecasts what educational goals shoxild be in the 
light of these probable future conditions (e.g., Rosove, I968; Weaver, 
1971) • Both types of forecasting were involved in establishing goals 
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for education in Atlanta, I985. The first type of .forecasting was accom- 
plished through tapping the perceptions of expe'rts through position papers 
they had written about the x^tiore in Georgia in their respective fields. 
The. second type of forecasting was accomplished through use of the Delphi 
technique. There is precedent for the use of the Delphi technique in 
forecasting what educational goals should be (e.g., see Cyphert and Gant, 
1970; and Uhl, 1971). <- 

The Delphi technique was developed by Rand Corporation for use in 
answering questions about the future when a great deal of uncertainty 
and complexity surround the area of concern (Dalkey, 1970). The procedure 
calls for iteration in eliciting perceptions from participants, so that 
they make a series of judgments, each successive one being made in the 
light of a summary of the judgments of all participants on the previous 
round. This process 'is designed to produce increasing accuracy of judg- . 
ment and increasing Agreement among participants from round to round. 
Rosove (1968), in evaluating ?1 different techniques for predicting the 
future, concluded tl^t the Delphi technique was among the five potentially 
most useful methods lof forecasting that might be applied to the functions 
of a center for educational policy research. The other four methods re- 
quire more information and more c^ertainty about the future than the Delphi 
technique does. Parenthetically, it may be noted that the study of educa- 
tional goals is a critical function of educational policy research. 

Convergence toward agreement among participants is considered to be 
of paramount importance in the use of the Delphi technique. . Convergence 
is the primary reason for employing an iterative process that is considerably 
more difficult to administer than a one-round survey, the traditional means 
of tapping perceptions. It may be anticipated that the more homogeneous in 



background the members of a Delphi panel are, the greater is the likeli- 
hood of convergence occxirring, given that the homogeneity is perceived 
by the members. Obversely, the more heterogeneous in background the 
members of a Delphi panel are, the less likely are they to converge. 
This relationship is readily derivable from a theoiy of cognitive balance 
as developed by a number of investigators (Heider, 1946; Osgood and 
Tannenbaum, 1955; Festinger, 1957). If it is assumed that use of the 
Delphi technique in educational goal-setting should involve large numbers 
of persons from a wide variety of backgrounds as participants, then the 
question of whether or not convergence among different groups of persons 
does in fact occur and, if so, to vrtiat extent, is a highly significant one. 

In the literature concerning use of the Delphi technique for educa- 
tional goal-setting, some attention has been given to convergence among 
groups as well as among individuals (e.g., Cyphert and Gant, 1970; and 
Uhl, 1971). Such studies have usually dealt with goal-setting in higher 
education in vdiich groups of participants were defined in terms of general 
role in respect to education, such as, faculty, administrators, students, 
organizational and political leaders, etc. Uhl reported convergence among 
such groups over rounds within the institutions of higher education studied. 
Cyphert and Gant did not report an analysis of convergence among groups, 
but they did report that differences in agreement within giroups were greater 
than those between groups. These investigators also made the suggestion, 
based upon anecdotal data from the written comments of respondents, that 
when respondents disagreed with the consensus rating of a goal, they tended 
to attribute that rating to a panel subgroup to which they did net belong. 
This tendency is consistent with the principle of cognitive balance referred 
to above. Cyphert and Gant reported further that the "University family" 



was theWoup making the "greatest change" in ratings throiighout the study. 
This groiap included facility members from the school of education, selected 
student ieaders, and top administrators and policy makers within the uni- 
versity. The University family, it may be assumed, was the most homogeneous 

of the several groups included in the study with respect to its subject, 

1 

which wasi desirable goals for the school of education. 

1 ■ • 

In tl^e present study, vdiich focused on student-centered goals of . 
public education at the secondary level in a major metropolitan area, the 
definition! of groups was not limited to role in respect to education. 
Groups beir^g examined were defined in part by race and sex. As integration 
is achievecj in school systems, the extent of agreement among groups from 
different racial backgrounds in respect to educational goals should be of 
some concern in establishing policy within a school system. Further^ with 
the current trend toward new conceptions of sex roles in our society, 
differences between males and females in the perception of educational 
goals are more important than ever. " 

Data Source . Three studies were conducted using the Delphi technique* 
One involved professional, technical, managerial, and commianity leaders in . 
the Atlanta area. The occupational divisions at the professional, technical, 
and managerial levels presented in the Dictionary of Occupational Titles 
(1965) were used heuristically for structuring the selection of respondents. 
Additionally, several leaders in the black community assisted in the identi- 
fication of blacks in the occupational and leadership categories to be 
represented. Several other categories of respondents were added to provide 
for individuals in public service roles that were primarily political in 
nature, e.g., members of the Atlanta Board of Education, members of the 
Atlanta Board of Aldermen, and state legislators from the Atlanta area. 



Cf the approximately 400 persons invited to participate in this study, 
275 completed all three roundk 

The second study involved high school teachers, counselors, principals, 
and other administrators directly involved with instruction in the Atlanta 
Public Schools. Teachers were selected to be representative of the entire 
range of subject matter in each of the 25 high schools then in the Atlanta 
system and also representative of the distribution of teachers by race and 
sex within each high school. All principals and other administrators that 
were directly involved with instruction were asked to participate. Of the 
445 that were invited to take part in the second study, 429 completed all 
three roimds. 

The third study involved high school student leaders selected to repre- 
sent the 25 high schools and the distribution of students by race and sex 
within each individual school. Of the 375 students invited to participate, 
369 completed all three roiinds. 

The Delphi technique has usually been employed with relatively small 
groups of participants. However, Cyphert and Gant (1970) and Uhl (l97l) 
report using much larger groups, 400 in the former study and almost 1,000 
in the latter. In the three studies reported here, a total of 1,073 res- 
pondents completed all three roimds. 

It has usually been the case that groups of experts have been impaneled 
as participants in a Delphi study. In both of the studies just cited, how^ 
ever, the expertise of respondents was de-emphasized. The results of 
investigation by Brown, Cochran, and Dalkey (1969), as reported by Uhl 
(1971), in which students were used as participants, sviggest that nothing 
of significance is lost by including less knowledgeable persons as long 
as there are some participants vrfio are knowledgeable. 
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Perhaps expertise is next a critical criterion for. selection of a 
respondent in a study that is concerned with what should be . Perhaps a 
more important question than who is expert is vrtiat kinds of persons shoxild 
be involved in deciding public policy. The question is as much political 
as technical, if not more so. Discussions of the accuracy of judgment 
(see Weaver, I97I) seem less applicable to the question of what shoxild be 
than to the question of what may be. 

Be that as it may, the three groups of respondents included in the 
Delphi studies in the Atlanta Assessment Project were perceived to have 
special areas of expertise related to education. It was felt that among 
the professional, technical, managerial, and community leaders of Atlanta 
resides the competence to make judgments about the relative importance of 
specific educational goals in the light of probable future conditions in 
the Atlanta area. It was thought that probably no group was more qualified 
than this one to make such judgments. Among the teachers and administrators 
of the Atlanta Public Schools resides another kind of expertise, an under- 
standing of the educational system and what it can do, and an understanding 
of students. Among the students resides a still different kind of expertise, 
for they are the ones vAio are living and experiencing the learning process. 
The student himself has perceptions of educational goals that, if for no 
other reason than his:, unique perspective as a learner, shoxild be included 
in a Delphi study of educational goals. 

Methods and Technicaies . The starting point in establishing educational 
goals for the Atlanta of 1985 was a set of S6 previously identified goals 
that had been adopted for the State as a whole by the Georgia Board of 
Education (Advisory Commission on Educational Goals, 1970). These goals 
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had been derived from position papers written about probable future con- 
ditions in the State by experts in a number of areas of concern (Schabieker- 
et al, 1970). A questionnaire designed to elicit a judgment about each of 
the 86 goals on a six-interval scale of importance was presented to each 
participant on three successive rovjids. Importance was considered in terms 
of preparing young people to live in the Atlanta of the future. In the 
first study, involving professional, technical, managerial, and community 
leaders, each respondent was interviewed personally once a week for three 
weeks. In the study involving students, the questionnaire was group-admini- 
stered eveiy two weeks over the three rounds. In the educator study, the 
questionnaire was handled in a variety of ways, all documented, from group 
administration to participant self-administration. What participants did 
in each of the three rounds in. evaluating goals is described below: 

Round One: To establish a future-oriented frame of reference in making 
judgments about the relative importance of goals, each participant was asked 
to read a short essay containing abstracts of the position papers that were 
used in the derivation of the goals. In responding to the questionnaire, 
each participant judged the relative importance of each of the goals in the 
questionnaire and then wrote down any additional goals that he felt were 
very important and should be included. 

Round Two: Each participant was given an opportunity to again read 
the essay containing the abstracts of the position papers about the future 
of Georgia if he so desired. Each participant responded to the same ques- 
tionnaire as in the first round, but with a difference. For each goal, 
the response category that was selected by the most participants in the 
first round — the modal response — was encircled. Participants were 
asked tto write in a "comments column" in the questionnaire their reasons 
for judging any particular goal to be either more important or less import 
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tant than the modal response. Additional goals suggested in Round 1 were 
submitted to participants in aii additional goals questionnaire that required 
judgments on the same scale of importance as that useji with the initial 86 
goals. 

Round Three: Each participant was again given an opportxpity to review 
the essay containing the abstracts of position papers about the futureXof 
Georgia if he so desired. The questionnaire used in the third round was 
the same as that used in the first two rounds, with appropriate response \ 
categories encircled to indicate the modal responses made in the second 
round. To further aid participants in making their final judgments, a 
summary of comments about each goal was presented with the questionnaire. 
This summary contained reasons given in Round 2 for judging each goal to 
be more important or less important than the modal response. An additional 
goals questionnaire was administered in Round 3 also. 
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Results and Conclusions , Analysis of data depended heavily upon 
nonparametric methods. For a general discussion of the techniques employed 
here, see SL^el (1956). Though similar analyses were performed on the 
additional goals, the results reported here are based on oiOLy the initial 
set of goals. 

The goals were rank ordered oh the basis of the mean importance of 
each goal as seen by community leaders, by educators, and by students 
respectively. An overall ranking was computed by taking the mean impor- 
tance rating across the three panels of responc?.ents for each goal and then 
ranking these. 

Each of the three panels of respondents— community leaders, educators, 
and students — was further broken down into four groups: white males, 
black males, white females, and black females. The goals were then further 
rank ordered on the basis of the perceived mean importance of each goal for 
each of these groups within the three panels. 

In ranking 86 goals on the basis of mean importance registered on a 
six-interval scale, the reliability of the ranking is a fundamental ques- 
tion. To determine reliability, each of the three panels was randomly 
divided into halves; and the goals were ranked separately for each half. 
The Spearman rank correlation technique was employed to determine the cor- 
relation in ranking between the halves of each group of participants. The 
resulting coefficients, computed for all three rounds, ranged from .96 to 
.99. 

A similar technique was employed to determine the reliability of the 
rankings by race and sex. Table 1 shows the coefficients for educators and 
students. The Delphi study of community leaders and the analysis of the 
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resulting data were coixiucted about five months earlier than that of 

educators and students, and the reliability of rankings by race and sesx for 

I 

community leaders was not computed. 

It can be* seen that the ranking of goals by race and sex tended to be 
highly reliable. Of the 24 coefficients reported, only one is below .90 
(•S4)# Table 2 shows the size of each gi-oup in each panel. 

To test for general convergence within each of the three Delphi 
panels, the Wilcoxon matched-pairs signed-ranks testj was used to determine 
whether the S. D,s became smaller from Rouixi 1 to Round 2 to Round 3. It 
was found uneqiiivocally that convergence did pccur on Round 2 for all three 
panels. The signed difference between the standard deviation of Round 1 
judgments and that of Round 2 judgments for each goal was positive in every 
case for each panel. 

On Round 3$ however, only the community leader panel showed general 
convergence. Both the student and educator panels showed considerable 
divergence. For the community leaders, the signed difference between the 
standard deviation of Round 2 judgments and that of Round 3 judgments for 
each goal was positive in 82 of the 86 cases. For educators, this differ- 
ence was negative in 66 of the 86 cases; and for students, it was negative 
in 70 of the 86 cases. The overall move(nent from Round 1 to Rouixi 3 for 
all three panels was nevertheless convergence. Divergence among students 
and educators on Round 3 will be considered in the discussion section of 
this paper. 

Convergence among groups defined in terms of race and sex within each 
Delphi panel was brought into focus by computing a Spearman rank correlation 
coefficient (rho) for each pair of groups within a panel for each round. 
Uhl (1971) used a similar technique in h^s investigation of the extent of 
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agreement among groups in a goal-setting Delphi study. Convergence amor^ 
groups did occur , demonstrated by the fact that, of the 18 possible pairings 
of groups within the three panels cn Round 2, l6 of them showed an increase 
in the correlation coefficient compared to Round 1, as shown in Table 3, 
and this occurred in spite of the relatively high level of agreement among 
groups at the outset. On Round 1 the average coefficient across ali pairs 
was ,88, as cOTpared to .93 on Round 2. ' - . 

There was no indication of further movement toward agreement among 
groups generally on Round 3. In fact, 7 of the 18 groups showed slight 
reductions in> the magnitude of the Spearman rho, whereas 6 groups showed 
slight increases, and 5 showed no change at all. 

' For all three panels, community leadei^s, educators, and students, 
the ^oups that tended to have the highest initial agreement, on Round 1, 
wer^ the. male and female groups of the same race, both black and white. 
The groups that tended to show the lowest level of agreement initially were 
black females with white males or females. Five of these six pairings 
(across the three panels) were initially as low as or lower than sny other 
pair within the particular panel. All six of these pairings showing low 
initial agreement showed an increase in agreement across rounds. 

In general, the educator panel showed higher initial agreement among 
groups and higher final agreement than did the other panels, perhaps due 
to the relatively greater commonality of orientation toward education that 
one would expect to find among educators as opposed to noneducators. 

To determine the degree of agreement among the three panels as ^a whole 
over rounds, the Spearman rank order correlation technique was again used. 
Table 4 shows these results. It may be seen that there was high initials 
agreement between the community leaders and educators and that the students 
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tended to be scxnewhat different from either of them. The general pattern 
of correlation coefficients shows clearly that inter-panel agreement de- 
clined on the second round, with a slight increase again on the third round. 
This finding will be considered in the discussion section of the paper. 

The rank ordering of goals on the basis of third-round judgments for 
each group within each panel and for each panel as a whole was examined 
in order to identify the particular goals representing the most critical 
or the most socially significant areasipf disagreement among groups. The 

> 

relative social significance of disag>eement was assumed to be a function 
of both the extent of disagreement and the relative importance of the goal 
at the heart of the disagreement. Extent of disagreement was operationally 
defined as the S. D. of the ranks assigned to a goal by the 12 groups within 
the three panels. A further index of disagreement examined was the range 
of the ranks assigned to a particular goal. As a measure of the relative 
importance of a given goal, the overall rank was used. 

Table 5 presents the ranks for each goal, group by group, and Table 6 
presents the S. D. of ranks, the range of the ranks, and the minimum and 
maximum ranks for each goal. To identify areas of criticalMisagreement, 
these two tables may be used conjunctively. Table 6 to identify the goals 
to be examined, and Table 5 to examine the actual ranks, group by group. 

For example, the first goal in Table "is able to listen, speak, 
read, and write," shows a pattern of ranks that may warrant examination 
because the goal is the most important one overall. Table 6 indicates that 
the S. D. of the ranks is 4.812, and that the ranks vary from 1 to 14. A 
closer look at Table 5 shows that five of the groups— all five being groups 
in the educator and community leader panels— gave a rank of 1 to the goal. 
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Three other groups gave this goal a rank of 2 or 3. Therefore, eighf of 
the groups were in fairly close agreement that this goal is of top iiiipor- 
tance. However, two groups, both black and white female students, gave 
this goal a rank of 14. The two remaining groups, white male students 
and educators, gave the goal a rank of 5 and 6 respectively. These dif- ' 
ferences were primarily between panels and between the sexes. 

It may be instructive to determine what black and white female students 
considered to be the most important goals. For black female students, the " 
top-ranked goal was, "is able to maintain individual integrity in group 
relationships." This goal, it may be seen, was the second-ranked goal for 
the student panel as a whole, whereas it was ranked 20bh by educators and 
28th by community leaders. For white female students, two goals Jtied for 
the top rank. They are, "supports the free and voluntary exercise of reli- 
gious choice," and, "understands freedom as the right to- make choices within 
the framework of concern for the general welfare." The key concept in these 
two goals is freedom of choice . 

Another example of a goal that would seem to warrant close examination 
along these lines is, ., "understands and accepts the responsibilities and privi- 
leges of citizenship." Community leaders ranked it 9th, as compared to a 
rank of 22 for educators, and a relatively low rank of 42 for students. The 
pattern of differences in ranking this goal has apparent implications for all ' 
three variables, panel, race, and sex. It may be seen in Table 6 that the S.D. 
among ranks for this goal is a relatively large 13.504, and the range is from 
a high of 1 to a low of 44. Within panels, the white female community leaders 
ranked this goal 1, as compared to 15 for black females. Among educators, 
there is the suggestion of a sex-related pattern of rankings, with both black 
and white males having given this goal a higher ranking than either the 
black or white female groups did. Among students both black males and females 
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ranked the goal lower than white males or females did. However, the greatest 
- differences in regard to this goal were between panels. 

An example of a goal for which the greatest differences were between 
groups within panels, rather than between the panels themselves, is the 
goal, "knows how and where to seek employment and "is able to apply for a 
job and participate in a job interview." This goal was ranked 20th by 
community leaders, 15th by educators, and l6th by students. However, within 
both the student and educator panels respectively, it can be seen that there 
were considerable differences between the races, with both black and white 
females having ranked the goal considerably higher than did the white male 
and female groups. In fact, for the black male students, this was the nvun- 
ber one goal. There is only the barest hint of a difference among groups 
in the community leader panel. 
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Discussion. There was definitely convergence among groups defined 
in tejgns of race and sex within each panel. Considering the relatively 
high initial agreement among the groups, it is impressive that any detec- 
table convergence occurred. That convergence between groups did occur • 
is testimony to the power of the Delphi technique in producing movement 
toward consensus, utilizdjig as it does the tendency toward cognitive 
balance. This tendency has been well documented as a powerful force in 
human behavior. 

iilhat happened on Round. 3 is of considerable interest. Looking at 
each panel as a whole, there was evidence of movement away from convergence 
in the educator and student panels. This finding is different from that 
reported by Cyphert and Gant (1970) and that reported by Uhl (1971). 
Cyphert and Gant found that movement on the last questionnaire of their 
study, which corresponded to Round 3 of the present investigation, was 
about equally divided between movement toward consensus and movement away 
from consensus, or in effect, no overall convergence at all. Uhl found 
that convergence did occur on Round 3, though it was not as marked as that 
on Round 2. ' 

A possible explanation for this tendency to diverge on Round 3 lies 
in the fact that this was the round on which a summary of dissenting opinions 
expressed on Round -2 was provided to each participant. It may be assumed 
that a summary of dissenting opiniohs would not encourage further convergence, 
but would in fact have the opposite effect, providing reinforcement for, a 
divergent response. This finding is particularly interesting in view of that 
reported by Sweigert and Schabacker (1974) regarding the inhibiting of con- 
vergence through feedback of each participant's own responses. Apparently 
the feedback of dissenting opinions, whether one's own or those of other 
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members of the panel, has an inhibiting effect on convergence. It wovad 
appear that the stronger effect in this connection is produced by exposure 
to thft dissenting opinions of others. Both of the two studies previously 
cited also included a summary of dissenting or minority opinions as feed- 
back to participants on the final round, but without the divergence effect 
found in the presert study, though Cyphert and Gant had hypothesized the 
occurrence of such an effect. 

A related finding of interest was that agreement among the three 
panels as a whole decreased on Round 2, as compared to Round 1, and tended 
to increase again on Round 3. In seeking an explanation for this finding, 
it should be kept in mind that the three "panels constituted groups completely 
independent of one another. Members of each panel received feedoack on the 
results of their own previous responses as a group, but did not receive feed- 
back on the responses of the other panels. Consequently, it may very likely 
have been the case that as the members of each panel converged among them- 
selves, one result was a slight reduction in the level of agreement across 
the panels. Further, it may have been the case that as the tendency to 
diverge occurred on Round 3, the level of agreement among the panels tended 
to increase slightly again. The changes in level of agreement over rounds 
was very small, but the pattern is consistent. 

In examining the ranks assigned to particxilar goals by the groups within 
each panel, it should be kept .in mind that there is definitely an element of 
judgment involved in identifying goals over which disagreement may be socially 
significant. The intent in this paper is to present a set of quantitative 
indices for social significance and to illustrate how these indices may be 
used. As pointed out in the previous section, the more important in general 
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a goal is perceived to be and the greater the variability among groups 
in perceiving its importance, the greater is the social significance of 
the disagreement about its importance and, consequently, the greater is 
the need to resolve the disagreement in setting policy. 

In the previous section, measures of the social significance of dis- 
agreement were provided for each goal examined in this study, and examples 
of different patterns of disagreement were presented. Since the groups 
were defined in terms of race, sex and overall panel, a pattern of dis- 
agreement might involve any one or any combination of these variables. 

A question might be raised as to why the rank ordering of goals was 
used rather than the actual mean ratings of importance on which the rank 
ordering was based. Ranks were used because the interest was in the rela- 
tive importance given to a particular goal by a group, not in the specific 
point on the importance scale constituting the arithmetic average of pe3>- 
ceptions. A rank ordering is of genuine practical significance because 
goals compete with one another for resources being allocated by a school 
system. Further, a rank ordering provides a kind of standard score, if you 
will, that tends to eliminate differences among groups in the use of scale 
points, i.e., where one group tends to give higher ratings generally than 
does another group. 

It should be kept in m^nd, of course, that use of a scale of importance 
such as this possibly has both "floor" and "ceiling" effects built into it. 
The top and bottom-ranked goals tended to have less variability in perceived 
importance than did the middle-ranked goals because there was less room to 
vaiy at the top and bottom than there was in the middle. This may be readily 
seen in Table 6. The floor and ceiling effects, if kept in mind, shoiad not 
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pose a serious problem iri using the indices* A six-interval scale is 
approaching the limit that can be used effectively in this type of 
measurement. (See Osgood, Suci, and Tamienbaum, I958.) Further, it may 
be contended tliat "tme" variability in responses may tend to decrease 
somewhat at either end of a scale because the most important and the least 
important are generally easier to identify than are things of only average 
importance. The middle range is a kind of nebulous area where discrimiaations 
tern' to be more difficxolt to make. Table 6 shows, however, that in spite 
of generally smaller S.D. 's at either end of the rank ordering^ differences 
in S.D. between goals that are adjacent to each^other in the ranking are 
frequently pronounced. These are the differences in variability that are 
worth examining. 

A question might be raised as to whether or not an examination of 86 
goals in a rank ordering may not risk capitalization on chance. In other 
words, how much confidence may one have that a pattern of rankings across 
groups is meaningful, and not Just a random occurrence. The answer to this 
question lies in' the stability of the rankings. The reliability coefficients 
for the rankings, shown in. Table 1, indicate that the rank orderin'', by groups 
was highly stable, making it generally unlikely that large differences in 
ranking a goal across groups were due* to chance. Ten of the twelve groups 
had N»s of very respectable size. Only the black and vAiite female groups 
in the community leader panel had N«s that were small enough to be bother- 
some. It perhaps should be stressed that the pattern of differences in 
ranking is of considerably more interest and importance than a single diff- 
erence between any two groups. A pattern is much less likely to be the 
result of chance than is a single difference. 
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Perhaps one final question should be considered. Drice a goal is identi- 
fied as having a socially significant level of disagreement among groups 
regarding its relative importance, what should be done with it? One reasonable 
approach would be to attempt to clarify the basis for the disagreement, per- 
haps through in-depth interviewing of selected panel members whose judgments 
reflect the different points of view, or perhaps through holding a meeting 
of selected panel members to discuss the nature of the disagreement. 

It is possible that. the disagreement might be the result of different 
interpretations of the goal statement, so that the resolution would be to 
develop a common imderstanding of what the goal statement means and possibly 
to generate new statements reflecting alternative interpretations. It is 
also possible that the disagreement is a genuine one, not merely differences 
in interpretation. Once the nature of the disagreement is understood, a 
decision can be made as to whether the goal needs to be restated, or split 
into more than one statement, or thrown out altogether, or dealt with in 
terms of its relevance to only part of the community, i.e., pluralistic 
sets of goals for a pluralistic community. 

Summary . In establishing educational goals through the Delphi tech- 
nique, three studies were conducted involving panels of community leaders, 
educators, and high school students respectively in metropolitan Atlanta. 
Convergence in perception of goals was examined within each panel as a whole 
a!nd among groups defined in terms of race and sex within panels. It was 
found that both individual and group perceptions generally tended to con- 
verge. Convergence among groups was particularly ijiipre^^ive because of 
the relatively high level of initial agreement among them. An additional 
finding of interest was the tendency for divergence on the third round 
among students and educators. Indices for identifying goals that reflect 
socially significant areas of disagreement' were presented. 
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Importance of the Study # There has been ijicreasing interest in the 

use of the Delphi teciinique in educational goal-setting* If it is assumed 

that use of the Delphi technique in educational goal-setting should involve 

large nuinoers of persons from a wide variety of ua'cKgrounds as participants, 

and if it is further asaumed that convergence among different groups of 

pers-ons participating in a Delphi study is important, then the question of 

whether or not convergence among different groups of persons does in fact 

occur is a highly significant one. In the present study, groups were defined 

in part in terms of race and" sex. As integration is achieved in school systems, 

the question of the extent of agreement among groups from different racial 

backgrounds as to what educational goals are important is of great concern in 

setting policy. Further, with the changing conceptions of sex roles in our 

» 

society, differences between males and female^ in the perception Olf the rela- 
tive importance of goals are of considerable importance. 

Even when there is relatively high agreement among groups, examination 
of .pecific areas where relative disagreement may exist be very useful. 
Where there is disagreement on a goal that is considered io be very important 
by one or more groups, there is a need for further analysis to determine the 
causes of disagreement. This kind of investigation may de-^elop information 
that has considerable relevance to policy setting within a school system. 
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TABLE 1 

RELIABILITY OF THE RANKINGS OF GOALS IN EACH ROUND 
BY RACE AND SEX, SHOWN FOR EDUCATORS AND STUDENTS 





Groxap 


Round 1 


Spearman Rank Correlation 
T?niinH 2 

1 iXj UI lU ^ 


PrtiinH 


Educators 








nilJLbc; rJcLLco 


01 


on 


.VO 


Black Males 


.93 


. .92 


.93 


Black Bemales 


.96 


.96 


.95 


White Females 


.94 


.94 


.97 


Students 








White Males 


.91 


.94 


.96 


Black Males 


.92 


.94 


.96 


Black Females 


.95 


.97 


.94 


White Females 


.84 


.94 


.91 



NOTE: In determini^ the reliability of the rankings, each group of participants 
was randomly divided into halves, and a ranking of goals was developed for each half. 
The correlation between the rankings for the halves was then computed for each group. 
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TABLE 2 

THE SIZE OF EACH GROUP IN EACH DELPHI PANEL 



Group N 



Conmunity Leaders 

White Males 153 

Black Males 83 

White Females 22 

Black Females 17 

Total 275 
Sttidents 

White Males 57 

Black Males 124 

White Females 53 

Black Females 135 

Total 369 
Educators 

White Ifales 82 

Black Males 98 

White: Females 111 
Black Females " 138 

Total 429 
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TABLE 3 



CORREIATIGNS AMMJG GROUPS DEFINED BY RACE AND SEX 
WITHIN EACH DELPHI PANEL IN RANKING THE 
GOALS FOR EDUCATION CN EACH ROUND 



Pairs of Groups 



Spearman Rank Correlation 
Rouna 1 Round 2 Round 3 



Community Leaders . 

White Males and Black Males 
White Males and White Females 
White Males and Black Females 
Black Males and Black Females 
Black Males and White Females 
Black Females and White Females 



.87 
.92 
.78 
.89 
.88 
.80 



.91 
.95 
.89 
.92 
.95 
.90 



.91 

.95 , 

.87 

.94 

.93 

.88 



Educators 



White 


Males and Black Males 


.87 


.93 


.94 


White 


Males and White Females 


.96 


.95 


.96 


White 


Males and Black Females 


.87 


.94 


.94 


Black 


Males and Black Females 


.95 


• .98 


.96 


Black 


Males and White Females 


.88 


.95 


.95 


Black 


Females and White Females 


.90 


.93 


.96 



Students 










White 


Males 


and 


Black Males 


. .85 


White 


Males 


and 


White Females 


.94 


, White 


Males 


and 


Black Females 


.83 


Black 


Males 


and 


Black Females 


• yO 


Black 


Males 


and 


White Females 




Black 


Females and White Females 


.8^ 



.93 
.95 
.60 
.98 
.95 
.93 



.91 
.96 
.91 
.97 
.91 
.93 
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TABLE 4 



CORRELATIONS AMONG THE THREE DELPHI PANELS IN RANKING 
THE GOALS FOR EDUCATION ON EACH ROUND 



Pairs of Groups 



Spearman Rank Correlation 
Roimd 1 Roimd 2 Roimd 3 



Coramiinity Leaders and Educators 
Community Leaders and Students 
Educators and Students 



.94 
.81 

.83 



.90 
.77 
.79 



.92 
.80 
.82 
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